In classification problems when multiples algorithms are applied to differentbenchmarks a difficult issue arises, i.e., how can we rank the algorithms? Inmachine learning it is common run the algorithms several times and then astatistic is calculated in terms of means and standard deviations. In order tocompare the performance of the algorithms, it is very common to employstatistical tests. However, these tests may also present limitations, sincethey consider only the means and not the standard deviations of the obtainedresults. In this paper, we present the so called A-TOPSIS, based on TOPSIS(Technique for Order Preference by Similarity to Ideal Solution), to solve theproblem of ranking and comparing classification algorithms in terms of meansand standard deviations. We use two case studies to illustrate the A-TOPSIS forranking classification algorithms and the results show the suitability ofA-TOPSIS to rank the algorithms. The presented approach is general and can beapplied to compare the performance of stochastic algorithms in machinelearning. Finally, to encourage researchers to use the A-TOPSIS for rankingalgorithms we also presented in this work an easy-to-use A-TOPSIS webframework.
展开▼
